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Abstract 



Reviews of 227 comparative studies and 50 descriptive studies covering the time periods 
1931-1999 and 1975-1999, respectively, revealed little change in response rates over time for 
postal mail surveys. Significant differences were found in comparative studies' response rates 
based on the academic area represented by the journal in which studies were published, with 
response rates higher for education and psychology journals than for business journals. Based on 
results of these reviews as well as a review of textbook advice to researchers, it is recommended 
that the minimum standard for response rate vary by population accessed but be set at 50% for 
business surveys and 70% for surveys in education or psychology. 
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Reviews of research studies investigating postal marl survey response rates have usually 
focused on identifying the survey methods most likely to produce high response rates. In order to 
obtain a body of studies large enough in number to examine the effects of different potential 
response facilitators it has been necessary to utilize studies conducted over a period of years, 
typically several decades. While some textbook authors, journal editors, and doctoral committees 
indicate expectations of acceptable response rates, there has been minimal research to establish the 
response rates generally obtained in mail surveys and whether these response rates are reasonable 
in today s climate. For academic and professional credibility, norms are sometimes used to 
evaluate what is and is not acceptable (e.g. values of Cronbach's alpha above .5; Nunnally, 1978), 
but norms for response rates are unclear. The purpose of this review was, first, to examine trends 
in response rates to published reports of mail surveys over time and, second, propose a standard for 
an acceptable response rate. 

Response rate to postal mail surveys has received extensive attention as failure to achieve 
an adequate response calls population parameter estimates into question. Potential demographic and 
interest differences between respondents and nonrespondents have lent impetus to response 
facilitation research. While we have continually advanced our knowledge of how to enhance 
response rates and have continually improved survey methodology, such advances may yield null 
results in light of reputed declines in participation in research in North America and Europe. 

Contradictions exist among the results of studies of research participation rates, with 
suggestions of declines in responses to in-person and phone surveys but less evidence of declines in 
postal mail responses. Steeh (1981) found a decrease in interview participation rates over the 
period from 1952 to 1979. Goyder (1986) found response rates to in-person surveys to be dropping 
in the United States and Canada. Sugiyama (1992) reported a similar trend for the Japanese as did 



Bethlehem and Kereten (1981) for the Dutch. Baim (1991) noted declines in response rates for the 
United Kingdom and France, with no declines for Switzerland and Germany. The reported decline 
was considered of sufficient importance to justify creation of an AAPOR task force in 1987 to 
research methods for reducing nonresponse. The Council for Marketing and Opinion Research 
(1998) suggested that "reversing or slowing the decline in respondent cooperation looms as one of 
the research industry's most urgent challenges" (p. 1). 

Less empirical documentation of trends in response rates to mail surveys over time is 
available, despite calls for such information (Steeh, 1981). Baruch (1998) cited a decline in 
response rate to behavioral science surveys from 1975 to 1995. He reviewed data in five 
management journals. Hox and De Leeuw (1994) located 45 studies that contrasted in-person, 
telephone, and postal mail survey response rates. Twenty-six comparisons with postal mail were 
identified in their data, which spanned the period from 1947 to 1992. Results with respect to in- 
person and phone surveys suggested declining participation while response rates to postal mail 
surveys were stable. It should be noted that their database was comparative studies, with a small 
number of studies (2-4) representing each five-year time span. Studies were drawn from the North 
American and western European research literature. Support for a stable pattern of mail survey 
response for populations in the Netherlands (De Leeuw, 1992) and Sweden (Lyberg & Lyberg, 
1990) were noted. Dillman and Carley-Baxter (2001) found no evidence of a decline in survey 
response rates from 1988 to 1999 in a yearly survey of visitors to United States national parks. In 
this study, however, surveys were delivered in-person and mailed back. 

The primary purpose of this study was to assess trends in response rate to postal mail 
surveys conducted in the United State over time. Two sources of empirical data were used to 
address this purpose. The first source was a database of 227 published articles documenting the 
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effects of experiments with mail survey response rate. The publication dates of the studies began 
with 1931 and continued to 1999. The second data source was published articles detailing the use 
of mail surveys in descriptive studies from 1975 through 1999. A random sample of 10 studies was 
selected for each five-year period yielding a database of 50 studies reporting 58 response rates- 
some studies reported more than one response rate. Because of differences in the two datasets 
(comparative studies from 1931 to 1999 in the first, descriptive studies from 1975 to 1999 in the 
second), information from the datasets were analyzed and reported independently. Comparative 
studies may not represent survey use by the wider community, so the smaller second database of 
descriptive studies was compiled to be more representative of general survey use. 

The secondary purpose of this study was to propose a general standard for response rate. A 
convenience sample of introductory research textbooks, survey research textbooks, and journal 
articles that stated recommendations for response rate were used to supplement the aggregate 
values from the two empirical databases. Differences in response rates by the academic area 
represented by journals were also assessed to determine if recommendations might differ by area of 
study. 

Method 

Comparative Studies. Studies were selected that met the following criteria; surveys were 
conducted via postal mail, the study reported results of a comparative or split-sample 
(experimental) approach, the survey was conducted in the United States and published in English, 
the response rate and sample size were reported, and the study report was available through 
published sources. Computer searches of four CD-ROM or on-line databases (ABI/Inform, 
PSYCHInfo, Sociofile, ERIC) were conducted using the search terms "mail survey" or "mail 
surveys combined with response rate" or "response rates." These sources were supplemented by 



a review of reference lists in published studies and of current issues of journals that consistently 
publish survey research studies. The database was originally compiled in 1994 and updated to 
December 1999. (See Boser and Clark, 1996, for citations through 1994 and the reference list in 

this paper for citations from 1995 through 1999.) Information from a total of 227 articles was 
abstracted. 

Information abstracted from each article included response rate, sample size, journal type, 
publication date, sampling technique, type of assignment to treatment, target population, whether 
follow-ups were used, and survey topic. Interrater agreement was assessed, disputes resolved, and 
re-assessed and deemed adequate. (See Green and Hutchinson, 1996, for a more complete 
description of the original database and method for assessing interrater agreement.) 

Studies were grouped by 5-year intervals with the exception of studies published prior to 
1949 where due to the small number of studies the interval was 1931-1949. Response rates were 
averaged across studies in each 5-year interval. Analysis of variance was used to determine if there 
were differences in mean response rates among the 5-year periods. The simple correlation of 
publication year with response rate was calculated. Then, standard multiple regression was used to 
predict response rate from abstracted variables and the significance of the coefficient for 
publication year examined. Differences in response rate by journal type (e.g., business, education) 
were also assessed, with the expectation that response rates would be significantly lower for 
business journals since business surveys tend to deal more with general populations than targeted 
professional groups. 

Descriptive Studies. Comparative studies were excluded from consideration for this 
database. The four computer-searchable databases described above were searched and 1 0 
published articles were randomly selected from each 5-year period beginning with 1975-1979, for a 



total of 50 articles. The same information was abstracted from these articles as described above, 
and studies were again grouped in 5-year intervals for analysis. Analysis of variance was used to 
assess changes in response rate over time. Differences in response rate by journal type were also 
assessed. 

Published Standards for Response Rate. A convenience sample of 14 social science, 
business, and survey research textbooks that contained response rate recommendations 
supplemented by journal articles and a dissertation was obtained and recommendations regarding 
response rate abstracted. 

Results 

Comparative Studies. Table 1 presents the average response rate by 5-year interval for 
comparative studies. No significant difference was found in response rate over time, F( 10,2 16) = 

1 .73, g=.08. The simple correlation of response rate with publication year was nonsignificant, r = - 
.04, p = .56. When response rate was predicted from publication date, journal type (business versus 
other), sampling technique (random versus other), topic (targeted versus not), assignment to 
treatment (random versus not), sample size, followup (yes versus no), and population description 
(general versus other), the multiple R was .35, g = .006. The regression coefficient for publication 
year was not significant, t = -.32, g = .75. The only significant predictor was followup, t = 1.99, g = 
.05. (When regression analyses were run separately for studies with and without use of followup, 
the multiple R’s were, respectively, R = .49 and R = .37.) In neither analysis was publication date a 
significant predictor. 

Table 2 displays response rate differentiated by journal type and in aggregate for 
comparative studies for 1995-1999. The assumption of homogeneity of variance was upheld, 
Levene's = 1.05, g = .37. Differences were significant at g.c.Ol (F 3>2 2 i = 5.83) for the aggregate 
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data. Tukey's honestly significant difference test revealed significant pairwise differences between 
the mean response rate for education compared to business, and education compared to "other. " It 
should be noted that there were only 7 studies abstracted between 1995 and 1999, with only one 
published in a non-business journal. 



Table 1 . Mean and Standard Deviation of Response Rate over Time for Comparative and 
Descriptive Studies 





Comparative Studies 


Descriptive Studies 


Time Period 


N 


Mean 


SD 


Range 


N 


Mean 


SD 


Range 


1931-1949 


6 


42.23 


16.10 


24.8-65.5 










1950-1954 


6 


34.57 


15.90 


19.5-61.1 










1955-1959 


5 


44.36 


12.82 


28.1-60.2 










1960-1964 


8 


44.28 


19.62 


21.0-82.8 










1965-1969 


15 


36.21 


11.70 


18.2-56.5 










1970-1974 


29 


48.56 


18.47 


17.5-80.5 










1975-1979 


45 


42.61 


^17.28 


18.3-85.2 


10 


69.69 


16.78 


42.0-95.0 


1980-1984 


30 


34.04 


13.75 


7.6-60.7 


12 


40.98 


15.15 


18.2-62.0 


1985-1989 


37 


45.06 


16.95 


13.5-79.0 


10 


50.23 


26.91 


14.6-93.0 


1990-1994 


39 


37.08 


20.83 


3.5-90.2 


11 


46.54 


22.59 


12.2-77.0 


1995-1999 


7 


38.21 


20.14 


7.1-69.2 


15 


54.57 


25.98 


12.8-96.5 


Total 


227 


41.00 


17.66 


3.5-90.2 


58 


52.09 


23.39 


12.2-96.5 
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Table 2. Mean, Standard Deviation, and Range of Response Rate by Journal Type for Comparative 
Studies. 





Aggregate 


1995-1999 


Journal Type 


N 


Mean 


SD 


Range 


N 


Mean 


SD 


Range 


Business 


131 


38.65 


16.29 


3.5-90.2 


6 


41.02 


16.85 


25.1-69.2 


Education 


37 


49.39 


16.61 


16.5-85.2 


1 


55.30 






Psychology 


33 


46.74 


18.93 


7.6-80.5 










Other 


24 


36.01 


17.53 


13.5-76.0 











Descriptive Studies. A significant difference in response rate over time was found for 
descriptive studies, F(4,53) = 2.56, p < .05. The homogeneity of variance assumption was upheld, 
Levene's = 2.08, p > .10. Tukey's honestly significant difference test revealed the single significant 
pairwise difference to be between the highest response rate in 1975-1979 and the lowest in 1980- 
1984 (Table 1). Mean response rates for 1985-1999 differed by 4-8%. 

Table 3 displays response rate differentiated by journal type and in aggregate for descriptive 
studies for 1995-1999. No differences significant at p < .05 by journal type were found for 1995- 
1999 or in aggregate. 

Table 4 displays recommendations for adequate response rates made by textbooks and 
journal authors. The minimum response rate recommended by survey research textbook authors is 
about 50% while the minimum response rate proposed by educational research authors seems 
closer to 70%. In business, recommendations vary by population addressed. 
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Table 3. Mean, Standard Deviation, and Range of Response Rate by Journal Type for Descriptive 
Studies. 







Aggregate 


1995-1999 


Journal Type 


N 


Mean 


SD 


Range 


N 


Mean 


SD 


Range 


Business 


17 


44.65 


24.31 


12.8-89.9 


4 


33.15 


22.16 


12.8-64.5 


Education 


5 


65.74 


10.08 


55.7-82.0 


2 


75.00 


9.90 


68.0-82.0 


Psychology 


16 


56.34 


22.87 


21.5-96.5 


4 


70.53 


28.23 


42.5-96.5 


Other 


20 


51.62 


24.38 


12.2-95.0 


5 


50.78 


20.79 


28.9-79.0 



Table 4. Response Rate Recommendations in Textbooks and Journals 



Textbooks 


Author 


Book Type 


Date 


Recommendation 


Aday 


Survey 


1996 


60-70% 


Babbie 


Survey 


1990 


50%-adequate, 60%-good, 70% very 
good 


Backstrom & Hursh-Cesar 


Survey 


1981 


70% is extraordinary but depends on 
type of survey 


Mangione 


Survey 


1995 


<50% not acceptable, 50-60 barely 
acceptable, 60-70% acceptable, 70-85% 
very good, 85% excellent 


Newman & McNeil 


Survey 


1998 


80% minimum or assessment of 
nonrespondent characteristics 


Rea & Parker 


Survey 


1997 


59-60% satisfactory 


Kervin 


Business 


1992 


50% typical; 60-70% good 


Pelosi, Sandifer, & Sekaran 


Business 


2001 


30% acceptable 


Gay 


Education 


1992 


60% is unacceptably low 


Johnson & Christensen 


Education 


2000 


Over 70% acceptable 


McMillan & Schumacher 


Education 


^2001 


=70% are doing very well 


Tuckman 


Education 


1999 


75-90% minimum 


Wiersma 


Education 


2000 


70% minimum with professional 
population; lower with general public 


Singleton, Straits & Straits 


Social 

Science 


1993 


50% minimal; above 65% quite good 


Journal Articles 


Anonymous 


Business 


1995 


Minimum 60% suggested by 
Advertising Research Foundation 


Baruch 


Business 


1999 


36%±13% for business managers; 
60%±20% other business oonulations 


Henderson 


Business 


1990 


20-30% for businesspersons 
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Discussion 



Findings of no response rate differences in comparative research studies and a 
nonsignificant increase in response rate differences in the last decade in descriptive studies support 
Lox and DeLeeuw's (1994) conclusion that response rates to postal mail surveys have remained 
substantially stable over time. This stability in response rate seems accompanied by downward 
trends in telephone and in-person interview participation rates. Frankel and Frankel (1987) argued 
that the problem in interview participation rates was exacerbated by changing lifestyles and 
demographic shifts, citing increased demands on individuals' time as one factor. De Maio (1980) 
four.-; 5 ’he top two reasons cited for interview refusal were invasion of privacy and negative past 
experiences with surveys. Schleifer (1986) found a downward trend in those reporting their last 
interview as a "pleasant" experience. Improvements in mail survey methodology and the relative 
convenience of mail survey response may have combined to counter the downward trend and keep 
response rates stable for this method. 

The low point in response rate in 1980-1984 could be due to publication of studies dealing 
with genera! populations, which tend to be placed in business journals. However, the proportion of 
the 22 studies from that time period that were published in business journals (17%) was lower than 
the proportion in 1975-1979 (30%). 

Significant differences in response rates across type of journal were found for comparative 
studies. Differences for descriptive studies were not significant, though the pattern found was 
similar to that found with comparative studies. As expected, response rates for studies published in 
business journals were the lowest on average and were significantly lower than response rates 
found on average in studies published in education journals. This finding reinforces the need to 
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consider characteristics of the population and whether the survey is targeted to that population in a 
priori estimation of response rates. 

It might be noted that response rates to comparative studies were lower in each time period 
than those of descriptive studies. In comparative studies, two conditions are of interest. One of 
those conditions is thought to be superior and to promote a higher response rate. The inclusion of 

one or more weaker conditions with lower response rates might tend to drag down the response rate 
means for those studies. 

The secondary purpose of this study was to propose a minimum standard for mail survey 
response rale. Typical average response rales in business are about 50 % with higher average rates 
in education and psychology (about 70%). These averages are in line with typical textbook 
suggestions thus it is suggested that standards for survey response, first, vary by population type 
and, second, be targeted at 50% ± 20% for business and 70% ± 20% for education and the social 

sciences. Thus the minimal standard would vary from 30% to 50% with ideal targets at 70% and 
90%. 

The results presented here are derived from a fairly lengthy history in postal mail survey 
research, ihe current trend, however, is toward use of web-based and e-mail surveys. If one 
considers web-based and e-mail surveys as more intrusive and less convenient for the respondent 
than postal surveys, it is likely that response rates to surveys in those modes may experience the 
same problems with participation that has been found with telephone surveys. A possible decline 
in responses to web- and e-mail surveys may be alleviated by improvements in convenience which 
are likely to accompany improvements in hardware and software. However, improved convenience 
might also be accompanied by an explosion in junk e-mail used for marketing that could alienate 
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respondents as it seems to have done with telephone surveys. The histoiy of e-mail use is brief, 
and it is unlikely that response rates will stabilize in the next several years. 
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